Data compression systems are known in the prior art that encode a stream of digital data signals into compressed digital code signals and decode the compressed digital code signals back into the original data. Data compression refers to any process that attempts to convert data in a given format into an alternative format requiring less space than the original. The objective of data compression systems is to effect a savings in the amount of storage required to hold or the amount of time required to transmit a given body of digital information.
To be of practical utility, a general purpose digital data compression system should satisfy certain criteria. The system should have reciprocity. In order for a data compression system to possess the property of reciprocity it must be possible to re-expand or decode the compressed data back into its original form without any alteration or loss of information. The decoded and original data must be identical and indistinguishable with respect to each other. The property of reciprocity is synonymous to that of strict noiselessness used in information theory. Some applications do not require strict adherence to the property of reciprocity. One such application in particular is when dealing with graphical data. Because the human eye is not that sensitive to noise, some alteration or loss of information during the compression de-compression process is acceptable.
The system should provide sufficient performance with respect to the data rates provided by and accepted by the devices with which the data compression and de-compression systems are communicating. The rate at which data can be compressed is determined by the input data processing rate into the compression system, typically in millions of bytes per second (megabytes/sec). Sufficient performance is necessary to maintain the data rates achieved in present day disk, tape and communication systems which rates typically exceed one megabyte/sec. Thus, the data compression and decompression system must have enough data bandwidth so as to not adversely affect the overall system. The performance of data compression and decompression systems is typically limited by the computations necessary to compress and de-compress and the speed of the system components such as, random access memory (RAM), and the like, utilized to store statistical data and guide the compression and de-compression process. Performance for a compression device is characterized by the number of processor cycles required per input character under the compressor. The fewer the number of cycles, the higher the performance.
Another important criteria in the design of data compression and de-compression systems is compression effectiveness, which is characterized by the compression ratio. The compression ratio is the ratio of data size in uncompressed form divided by the size in compressed form. In order for data to be compressible, the data must contain redundancy. Compression effectiveness is determined by how effectively the compression procedure uses the redundancy in the input data. In typical computer stored data, redundancy occurs both in the nonuniform usage of individual symbology, example digits, bytes, or characters, and in frequent recurrence of symbol sequences, such as common words, blank record fields and the like.
General purpose data compression procedures are also known in the prior art, three relevant procedures being the Huffman method, the Tunstall method and the Lempel-Ziv method. The Huffman method is widely known and used, reference thereto in article of D. A. Huffman entitled "A Method For Construction Of Minimum Redundancy Codes", Proceedings IRE, 40, 10 pages 1098-1100 (September 1952). Reference to the Tunstall algorithm may be found in Doctoral thesis of B. P. Tunstall entitled "Synthesis of Noiseless Compression Codes", Georgia Institute of Technology (September 1967). Reference may be had to the Lempel-Ziv procedure in a paper authored by J. Ziv and A. Lempel entitled "A Universal Algorithm For Sequential Data Compression", IEEE Transactions on Information Theory, IT-23, 3, pages 337-343 (May, 1977).
One of the first general purpose data compression procedures developed is the Huffman method. Briefly described, the Huffman procedure maps full length segments of symbols into variable length words. The Huffman data compression procedure suffers from two limitations. Firstly, the Huffman procedure operates under the constraint that the input data to be compressed be parsed into fixed length segments of symbols. Although the Huffman procedure provides the best compression ratio that can be obtained under these constraints, when the constraint is relaxed it is possible to obtain significantly better compression ratios by utilizing other procedures. Secondly, Huffman coding requires full knowledge of the statistical characteristic of the source data. The Huffman procedure operates under the assumption that the probability with which each fixed length input segment occurs is known. This requirement of the Huffman procedure can in practice, be satisfied by the use of an adaptive version of the procedure which accumulates the necessary statistics during processing of the data. This, however, is cumbersome, and requires considerable working memory space and performs sub-optimally during adaptation.
The Tunstall algorithm, which maps variable length segments of symbols into fixed length binary words, is complimentary to the Huffman procedure with the fixed length constraints now applied to the output segments instead of the input segments. Like the Huffman procedure, the Tunstall procedure requires a foreknowledge of the source data probabilities. Again this foreknowledge requirement can be satisfied to some degree by utilizing an adaptive version which accumulates the statistics during processing of the data.
The Lempel-Ziv procedure maps variable length segments of the symbols into variable length binary words. It is asymptotically optimal when there are no constraints on the input or output segments. In this procedure the input data string is parsed into adaptively grown segments, each segment consisting of an exact copy of an earlier portion of the input string suffixed by one new symbol from the input data. The copy which is to be made is the longest possible and is not constrained to coincide with any earlier parsed segment. The code word which replaces the segment in the output contains information consisting of a pointer to where the earlier copied portion begins, the length of the copy, and the new symbol.
It would appear that Huffman or Shannon-Fano coding is the perfect means of compressing data. However, this is not the case. As mentioned above, these coding methods are optimal when and only when the symbol probabilities are integral powers of 1/2, which is usually not the case.
The technique of arithmetic coding does not have this restriction: It achieves the same effect as treating the message as one single unit (a technique which would, for Huffman coding, require enumeration of every single possible message), and thus attains the theoretical entropy bound to compression efficiency for any source.
In arithmetic coding, one decision after another is encoded to define successfully smaller, lesser included intervals along a number line. Additional information on arithmetic coding can be found in "An Introduction To Arithmetic Encoding"; by G. G. Langdon, Jr., IBM Journal of Research and Development, Vol. 28, n. 2, March 1984, 135-149; and "Arithmetic Compression Code Control Parameters Approximation"; by D. R. Helman, G. G. Langdon, Jr., and J. J. Rissanen, Vol. 23, n. 11, April 1981, 5112-5114 and U.S. Pat. No. 4,905,297, "Arithmetic Coding Encoder And Decoder System", Langdon, Jr. et al. all incorporated herein by reference.
As noted in the above articles, arithmetic coding provides that each decision has a plurality of possible exclusive outcomes "or events". Each outcome or event is represented in data by a symbol. In the imaging environment, for example, each decision may correspond to whether or not a given pixel is black. The decision outcome being represented by a Y (or YES) symbol if the pixel is black or an N (or NO) symbol if the pixel is not black. Thus, a plurality of decisions may then be represented by a sequence of symbols, e.g., YNNY . . . .
In accordance with prior arithmetic coding teachings, a probability line has a current interval defined thereon. The first current interval is 0 to 1. The current interval is divided into segments in which segment corresponds to one possible outcome for the next decision. Where there are only two possible outcomes for each decision, the current interval is divided into two segments. The length of each segment is based on its respective associated probability. The respective probabilities may remain fixed or may adapt as decision data is entered.
It is the correlating of large segments to symbols which occur with greater frequency that leads to the compression effect. In the former cited article ("An Introduction To Arithmetic Encoding"), a 4-symbol arithmetic coding example is set forth in which each decision can result in an "a" event (having a 50% probability), a "b" event (having a 25% probability), a "c" event (having a 12.5% probability), or a "d" event (having a 12.5% probability). Representing the four events in binary form would require two bits for each decision, where the events would be represented respectively by 00, 01, 10, 11. For three decisions such as "aab" which is highly likely, the straight forward encoding data would be 00 00 01; requiring six bits. However, as observed in the article at page 137, the arithmetic coding approach permits the sequence "aab" to be represented by the value 0.001. Instead of six bits, the information can be represented in three bits. This conservation of bits results as successive events having relatively high associated probabilities occur.
The conservation deteriorates if numerous events occur for which there are low probabilities and relatively short line segments. With the above noted probabilities, a sequence of events "dd" would be represented with encoded data as 11 11 whereas, by arithmetic coding, the "dd" events would be represented by 111111. Provided that the larger segments in fact correspond to events that occur with correspondingly greater frequency, the additional bits needed for less probable symbols are outweighed by the conservation achieved when more probable symbols occur.
When looking for the best lossless compression ratios, arithmetic coding is the state of the art, especially when considering adaptive algorithms. When the arithmetic coder is given a good context model, the compression ratios improve from 200% to 300% over other available LZ based techniques. Additionally, in many cases where the data is nearly random (multi-bit data/photographs), arithmetic coding can obtain a 20% compression when LZ would expand the image.
Memory has become a problem for many systems. In particular, in a laser printer system, 4 MB of RAM is now required to allow the printer to handle 600 DPI data at 12 ppm. The next generation products will have 32 ppm engines with 600 DPI data at 2 bits/pixel. The amount of data has more than doubled, but the memory constraint of 4 MB is constant. The reason the data has more than doubled is due to the page pipe architecture needed to keep several pages in the printer at once. While the current page is very nearly through the printer, the next page begins to move. Descriptions of both pages are needed to keep the printer streaming and still allow for recovery from paper jams or other errors.
Perhaps one of the most serious drawbacks of arithmetic coding is the data throughput performance. Some significantly complicated logic is required to sustain just one bit/cycle performance. This has created a dilemma for the imaging systems because the following constraints have diverged:
Lossless Compression Required for Fidelity in Most Areas PA1 Algorithm Must Adapt to Many Forms of Data PA1 High Compression Ratios--More Data in Systems, Cost Pressure PA1 Higher Data Throughput/Data Needed Real Time
It is presently possible, but difficult to produce arithmetic coding hardware capable of operating at a 66 MHz rate. This yields between 6.0 MB/s to 7.5 MB/s data rates when using the standard one bit at a time approach. The next generation printers need data at a rate of 5.5 MB/s. Real time constraints and the overhead associated with controlling the hardware compressor indicate that the single bit context model is not acceptable. A minimum of 9 MB/s is required to allow for this overhead.